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We evaluated a staff training and management package for increasing accuracy of recording 
frequency of problem behavior in a residential care facility. A multiple baseline design across the 
first and second work shifts showed that 2 of 8 participants increased their accuracy following in- 
service training, and all 8 improved during a condition with supervisor presence and feedback. 
Improvements were maintained when feedback was removed and generalized to activity periods 
when neither supervisor presence nor feedback was provided. Other staff behavior was not 
adversely affected by the intervention package. 
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A cornerstone of applied behavior analysis is 
precise data collection to support objective 
evaluation of interventions in educational and 
human service environments (Baer, Wolf, & 
Risley, 1968). However, little research is available 
on the reliability of data obtained from staff who 
provide ongoing service delivery in community 
settings. To address this issue, we assessed staff 
accuracy in recording frequency of problem 
behavior displayed by persons with developmen- 
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tal disabilities in a residential care facility. We 
then examined whether accuracy improved with 
standard behavior-analytic methods for improv- 
ing staff performance (in-service training fol- 
lowed by performance feedback from super- 
visors; Reid, 1998). We then conducted probes 
to evaluate generalization and maintenance. 

METHOD 

Participants and Setting 

Participants were 8 paraprofessional direct- 
care instructors (DCI) in a residential treatment 
setting for persons with developmental disabili- 
ties. DCIs were women (except BD), aged 22 to 
60 years old, with 1 month to 11 years 
9 months experience in developmental disabili- 
ties. Each DCI worked during either the first or 
second shift, and each was assigned to either 
Treatment Group 1 (3 women with profound 
mental retardation) or Group 2 (2 men with 
profound mental retardation). 
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Design and Procedure 

A multiple baseline design was applied across 
staff on the first and second shifts in each 
treatment group. During baseline, DCIs fol- 
lowed their usual routines. Supervisors, who 
included the senior behavior analyst (the first 
author) and two female DCI supervisors, were 
present and gave performance feedback, al- 
though they did not follow a specific protocol 
for doing so. 

The first intervention was a 45- to 60-min 
in-service training session that consisted of 
a lecture by the first author on frequency data 
collection, followed by distribution of materials 
to ensure that each DCI had one pouch card 
with accurate definitions and space to enter data 
for each person with target problem behaviors. 
Each card was 8 cm by 13 cm; data were 
recorded on the occurrence of a behavior and 
the time (indicated by wall-mounted digital 
clocks with large numbers). 

The second intervention was a systematic 
procedure for supervisor presence and feedback. 
At the onset of each session, the supervisor 
entered the observation area, observed for 
6 min, and recorded each target behavior that 
occurred on his or her own set of pouch cards. 
The supervisor then approached the DCI one at 
a time, compared the DCI’s data entries to his 
or her own entries and gave feedback, including 
(a) a statement of agreement that the target 
behavior did or did not occur, (b) praise for 
agreements, (c) prompts to practice by copying 
the supervisor’s entry if there were disagree- 
ments, and (d) reminders to continue to make 
entries throughout the work shift. The final 
condition involved supervisor presence without 
feedback. During this condition the supervisor 
entered the observation area, made entries on 
his or her pouch cards, and left at the end of 
6 min without providing feedback. 

Measurement and Interohserver Agreement 

Trained student research assistants served as 
observers, a common practice in the setting that 
may have reduced staff reactivity. The main 


dependent measure was percentage agreement 
between each DCI and observers regarding the 
occurrence of target problem behavior, in- 
cluding its time (to the minute), code, and 
number of tallies. Target problem behavior 
included physical and verbal aggression, self- 
injury, property destruction, disrobing, and 
pica. Data were also collected to confirm that 
the intervention did not have adverse effects on 
other staff duties and to verify that supervisors 
adhered to observation and feedback procedures 
(details available from the first author). 

Primary observations of DCI data collection 
were conducted 4 to 5 days per week in 
classrooms and living, dining, and recreation 
rooms. Generalization observations, during 
which supervisors were absent, were performed 
2 to 3 days per week in the hour following 
primary observations for Group 1 (second 
shift), 3 hr later for Group 2 (second shift), 
and 4 hr later for Groups 1 and 2 (first shift) in 
living, dining, and recreation rooms. All 
observations were 55 min long. Agreement 
between independent observers was assessed in 
29% of sessions; the mean was 90% (range, 0% 
to 100% across sessions). 

RESULTS AND DISCUSSION 

The top two graphs of Figure 1 depict the 
percentage of problem behaviors recorded by 
individual DCIs in each of the two shifts, for 
each of the two treatment groups, during 
primary observation sessions. Data points 
represent DCI performance on days when 
target problem behaviors occurred, providing 
the opportunity to make frequency recordings. 
When 2 DCIs were present, data are shown for 
each. The mean number of target behaviors per 
session ranged from 1 to 4.3 across conditions 
for all shifts except Group 1 (second shift). This 
shift encountered problem behavior more often, 
with means ranging from 4.5 to 15 across 
conditions. 

The results show that the in-service training 
had no effect on 5 of the 8 DCIs’ frequency- 
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Figure 1. The top two graphs depict percentage of target problem behaviors recorded by direct-care instructors for 
Groups 1 and 2 across first and second shifts in primary observations. Data points represent sessions in which target 
problem behaviors occurred. The bottom graph depicts performance in generalization sessions; the numbers on top of 
each bar indicate the number of sessions in which target problem behaviors occurred. 
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recording performance (there was no opportu- 
nity to assess AB’s performance in this 
condition). Supervisor presence and feedback 
led to immediate and marked improvements for 
those DCIs who did not respond to the initial 
training. The improvement in JA’s frequency 
recording following the in-service training was 
maintained in the condition with supervisor 
presence and feedback, and AW showed 
additional improvement over that obtained 
with the in-service training. Improvements 
appeared to be maintained and perhaps in- 
creased during supervisor presence without 
feedback, although additional data are needed 
to confirm this finding. Group 1 (second shift) 
showed the greatest fluctuation in performance 
in the supervision conditions, perhaps due to 
the higher rate of problem behaviors that the 
group had to record. 

The bottom graph in Figure 1 displays 
generalization data for those DCIs who had 
opportunities to respond in baseline and at least 
two experimental conditions. The number of 
sessions when problem behavior occurred is 
depicted above each bar. The data suggest that 
staff generalized accurate data collection to 
settings in which supervisors were absent, but 
this conclusion must be made with caution 
because of the small number of data points. 

One implication of the findings is that the 
supervisor presence already inherent in the 
treatment environment did not lead to adequate 
frequency recording. Fience, the introduction of 
staff management techniques was apparently 
necessary. Also, consistent with other research 
(e.g., Gardner, 1972), in-service training by 
itself was ineffective in improving performance, 
suggesting that skill deficits alone were not the 
major factor that contributed to DGI baseline 
or post-in-service performance. 

DGI performance improved substantially 
with a very brief supervisory period (6 min) 
followed by feedback. Thus, most frequency 
recording occurred when supervisors were not 


present. Further, improvements were main- 
tained even after feedback was abruptly with- 
drawn and may have generalized to sessions 
when supervisory techniques were never ap- 
plied. Moreover, because target problem behav- 
iors occurred in a minority of observation 
intervals, supervisor feedback regarding the 
nonoccurrence of problem behavior seems to 
have been enough to increase DGI frequency 
recording. Finally, it appears that high rates of 
problem behaviors (Group 1, second shift) 
resulted in more variable staff performance, 
suggesting the need for enhanced supervision 
(e.g., longer monitoring periods) or alternative 
methods of measuring problem behavior in 
such cases. 

Supervisory techniques in other settings in 
which behavioral data are collected (e.g., class- 
rooms) and with different types of data 
collection warrant further investigation. Anoth- 
er topic for research is whether in-service 
training is necessary or whether supervision 
and feedback alone are effective in improving 
staff performance. Nevertheless, the supervision 
methods in this study may be an efficient means 
of improving and maintaining staff recording of 
problem behavior, which may, in turn, support 
sounder evaluations of services for people with 
developmental disabilities. 
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